{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Australian Geoscience Datacube API\n",
    "This notebook describes connecting to the datacube and doing a basic query"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import datacube.api\n",
    "from pprint import pprint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default, the API will use the configured database connection found in the config file.\n",
    "\n",
    "Details on setting up the config file and database and be found here:\n",
    "http://agdc-v2.readthedocs.org/en/develop/db_setup.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dc = datacube.api.API()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary functions\n",
    "* __`list_fields()`__ - lists all fields that can be used for searching\n",
    "* __`list_field_values(field)`__ - lists all the values of the field found in the database\n",
    "\n",
    "Find out what fields we can search:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['gsi', 'sat_path', 'product', 'id', 'collection', 'time', 'platform', 'lat', 'orbit', 'sat_row', 'lon', 'instrument'])"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dc.list_fields()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `product` and `platform` fields looks interesting. Find out more about them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['gamma0']"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dc.list_field_values('product')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['SENTINEL_1A', 'ALOS_2']"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dc.list_field_values('platform')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Query and Access functions\n",
    "There are several API calls the describe and provide data in different ways:\n",
    "\n",
    "* __`get_descriptor()`__ - provides a descripton of the data for a given query\n",
    "* __`get_data()`__ - provides the data as `xarray.DataArray`s for each variable.  This is usually called based on information returned by the `get_descriptor` call.\n",
    "* __`get_data_array()`__ - returns an `xarray.DataArray` n-dimensional object, with the variables stack along the dimension labelled `variables`.\n",
    "* __`get_dataset()`__ - return an `xarray.Dataset` object, containing an `xarray.DataArray` for each variable."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  get_descriptor\n",
    "We can make a query and find out about the data:\n",
    "\n",
    "The query is a nested dict of variables of terms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'alos2_gamma0_albers': {'dimensions': ['time', 'y', 'x'],\n",
      "                         'irregular_indices': {'time': array(['2016-03-02T10:59:59.000000000+1100'], dtype='datetime64[ns]')},\n",
      "                         'result_max': (numpy.datetime64('2016-03-02T10:59:59.000000000+1100'),\n",
      "                                        -4300006.25,\n",
      "                                        1499993.75),\n",
      "                         'result_min': (numpy.datetime64('2016-03-02T10:59:59.000000000+1100'),\n",
      "                                        -4899993.75,\n",
      "                                        900006.25),\n",
      "                         'result_shape': (1, 48000, 48000),\n",
      "                         'variables': {'hh_gamma0': {'datatype_name': dtype('float32'),\n",
      "                                                     'nodata_value': 0},\n",
      "                                       'hv_gamma0': {'datatype_name': dtype('float32'),\n",
      "                                                     'nodata_value': 0}}},\n",
      " 's1_gamma0_albers': {'dimensions': ['time', 'y', 'x'],\n",
      "                      'irregular_indices': {'time': array(['2016-03-02T10:59:59.000000000+1100'], dtype='datetime64[ns]')},\n",
      "                      'result_max': (numpy.datetime64('2016-03-02T10:59:59.000000000+1100'),\n",
      "                                     -4300006.25,\n",
      "                                     1499993.75),\n",
      "                      'result_min': (numpy.datetime64('2016-03-02T10:59:59.000000000+1100'),\n",
      "                                     -4899993.75,\n",
      "                                     900006.25),\n",
      "                      'result_shape': (1, 48000, 48000),\n",
      "                      'variables': {'vh_gamma0': {'datatype_name': dtype('float32'),\n",
      "                                                  'nodata_value': 0},\n",
      "                                    'vv_gamma0': {'datatype_name': dtype('float32'),\n",
      "                                                  'nodata_value': 0}}}}\n"
     ]
    }
   ],
   "source": [
    "query = {\n",
    "    'product': 'gamma0',\n",
    "    'platform': ['ALOS_2','SENTINEL_1A'],\n",
    "}\n",
    "descriptor = dc.get_descriptor(query, include_storage_units=False)\n",
    "pprint(descriptor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The query can be restricted to provide information on particular range along a dimension.\n",
    "\n",
    "For spatial queries, the dimension names should be used.  The default projection for the range query values is in WGS84, although"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'alos2_gamma0_albers': {'dimensions': ['time', 'y', 'x'],\n",
      "                         'irregular_indices': {'time': array(['2016-03-02T10:59:59.000000000+1100'], dtype='datetime64[ns]')},\n",
      "                         'result_max': (numpy.datetime64('2016-03-02T10:59:59.000000000+1100'),\n",
      "                                        -4548968.75,\n",
      "                                        1284918.75),\n",
      "                         'result_min': (numpy.datetime64('2016-03-02T10:59:59.000000000+1100'),\n",
      "                                        -4666481.25,\n",
      "                                        1187756.25),\n",
      "                         'result_shape': (1, 9402, 7774),\n",
      "                         'variables': {'hh_gamma0': {'datatype_name': dtype('float32'),\n",
      "                                                     'nodata_value': 0},\n",
      "                                       'hv_gamma0': {'datatype_name': dtype('float32'),\n",
      "                                                     'nodata_value': 0}}},\n",
      " 's1_gamma0_albers': {'dimensions': ['time', 'y', 'x'],\n",
      "                      'irregular_indices': {'time': array(['2016-03-02T10:59:59.000000000+1100'], dtype='datetime64[ns]')},\n",
      "                      'result_max': (numpy.datetime64('2016-03-02T10:59:59.000000000+1100'),\n",
      "                                     -4548968.75,\n",
      "                                     1284918.75),\n",
      "                      'result_min': (numpy.datetime64('2016-03-02T10:59:59.000000000+1100'),\n",
      "                                     -4666481.25,\n",
      "                                     1187756.25),\n",
      "                      'result_shape': (1, 9402, 7774),\n",
      "                      'variables': {'vh_gamma0': {'datatype_name': dtype('float32'),\n",
      "                                                  'nodata_value': 0},\n",
      "                                    'vv_gamma0': {'datatype_name': dtype('float32'),\n",
      "                                                  'nodata_value': 0}}}}\n"
     ]
    }
   ],
   "source": [
    "query = {\n",
    "    'product': 'gamma0',\n",
    "    'platform': ['ALOS_2','SENTINEL_1A'],\n",
    "    'dimensions': {\n",
    "        'x' : {\n",
    "            'range': (146.0, 147.0),\n",
    "        },\n",
    "        'y' : {\n",
    "            'range': (-42.0, -41.0),\n",
    "        },\n",
    "        'time': {\n",
    "            'range': ((2015, 1, 1), (2017, 1 ,2)),\n",
    "        }\n",
    "    }\n",
    "}\n",
    "pprint(dc.get_descriptor(query, include_storage_units=False))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A coordinate reference sytsem can be provided for the spatial dimensions, either as a EPSG code or a WKT description:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "query = {\n",
    "    'product': 'gamma0',\n",
    "    'platform': ['ALOS_2','SENTINEL_1A'],\n",
    "    'dimensions': {\n",
    "        'x' : {\n",
    "            'range': (1187756.25, 1284918.75),\n",
    "            'crs': 'EPSG:3577',\n",
    "        },\n",
    "        'y' : {\n",
    "            'range': (-4666481.25,-4548968.75),\n",
    "            'crs': 'EPSG:3577',\n",
    "        },\n",
    "        'time': {\n",
    "            'range': ((2016, 1, 1), (2017, 1 ,1)),\n",
    "        }\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### get_data\n",
    "This retrieves the data, usually as a subset, based on the information provided by the `get_descriptor` call.\n",
    "\n",
    "The query is in a similar form to the `get_descriptor` call, with the addition of a `variables` parameter.  If not specified, all variables are returned.\n",
    "The query also accepts an `array_range` parameter on a dimension that provides a subset based on array indicies, rather than labelled coordinates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['indices', 'size', 'arrays', 'dimensions', 'element_sizes', 'coordinate_reference_systems'])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "query = {\n",
    "    'product': 'gamma0',\n",
    "    'platform': 'ALOS_2',\n",
    "    'variables': ['hh_gamma0', 'hv_gamma0'],\n",
    "    'dimensions': {\n",
    "        'x' : {\n",
    "            'range': (146, 147),\n",
    "            'array_range': (0, 1),\n",
    "        },\n",
    "        'y' : {\n",
    "            'range': (-41, -42),\n",
    "            'array_range': (0, 1),\n",
    "        },\n",
    "        'time': {\n",
    "            'range': ((2016, 1, 1), (2017, 1, 1))\n",
    "        }\n",
    "    }\n",
    "}\n",
    "data = dc.get_data(query)\n",
    "data.keys()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### get_data_array\n",
    "This is a convinence function that wraps the `get_data` function, returning only the data, stacked in a single `xarray.DataArray`.\n",
    "\n",
    "The variables are stacked along the `variable` dimension."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "alos2 = dc.get_data_array(product='gamma0', platform='ALOS_2', y=(-41,-42), x=(146,147))\n",
    "s1a = dc.get_data_array(product='gamma0', platform='SENTINEL_1A', y=(-41,-42), x=(146,147))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### get_dataset\n",
    "This is a convenience fuction similar to `get_data_array`, returning the data of the query as a `xarray.Dataset` object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<xarray.Dataset>\n",
       "Dimensions:    (time: 1, x: 7774, y: 9402)\n",
       "Coordinates:\n",
       "  * time       (time) datetime64[ns] 2016-03-01T23:59:59\n",
       "  * y          (y) float64 -4.549e+06 -4.549e+06 -4.549e+06 -4.549e+06 ...\n",
       "  * x          (x) float64 1.188e+06 1.188e+06 1.188e+06 1.188e+06 1.188e+06 ...\n",
       "Data variables:\n",
       "    crs        int32 0\n",
       "    vh_gamma0  (time, y, x) float32 nan nan nan nan nan nan nan nan nan nan ...\n",
       "    vv_gamma0  (time, y, x) float32 nan nan nan nan nan nan nan nan nan nan ...\n",
       "Attributes:\n",
       "    title: Experimental Data files From the Australian Geoscience Data Cube - DO NOT USE\n",
       "    license: Creative Commons Attribution 4.0 International CC BY 4.0\n",
       "    product_version: 0.0.0\n",
       "    source: This data is a reprojection and retile of Landsat surface reflectance data from the USGS\n",
       "    summary: These files are experimental, short lived, and the format will change."
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dc.get_dataset(product='gamma0', platform='SENTINEL_1A', y=(-41,-42), x=(146,147))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
